Add migrating-newtonsoft-to-system-text-json skill#200
Add migrating-newtonsoft-to-system-text-json skill#200mrsharm wants to merge 7 commits intodotnet:mainfrom
Conversation
There was a problem hiding this comment.
Pull request overview
Adds a new .NET skill intended to guide migrations from Newtonsoft.Json (Json.NET) to System.Text.Json, plus an evaluation scenario to validate the skill’s impact.
Changes:
- Added a new skill doc covering attribute mappings, behavioral differences, converter migration, and DOM migration guidance.
- Added a new eval scenario for migrating a Newtonsoft-attributed model and configuring global JSON options.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 11 comments.
| File | Description |
|---|---|
| src/dotnet/skills/migrating-newtonsoft-to-system-text-json/SKILL.md | New skill documentation for Newtonsoft.Json → System.Text.Json migration guidance |
| src/dotnet/tests/migrating-newtonsoft-to-system-text-json/eval.yaml | New evaluation scenario exercising attribute/config migration guidance |
Comments suppressed due to low confidence (2)
src/dotnet/skills/migrating-newtonsoft-to-system-text-json/SKILL.md:208
- The
.csprojsnippet usesVersion="*", which isn't a valid/typicalPackageReferenceversion and may confuse readers. Since the intent is "remove these references", consider omitting theVersionattribute entirely or showing a realistic pinned/floating version format (e.g.,13.0.3or13.*) in examples elsewhere.
```xml
<!-- Remove from .csproj -->
<PackageReference Include="Newtonsoft.Json" Version="*" />
<PackageReference Include="Microsoft.AspNetCore.Mvc.NewtonsoftJson" Version="*" />
src/dotnet/tests/migrating-newtonsoft-to-system-text-json/eval.yaml:5
- The prompt says the model "uses ... a custom converter", but the snippet only uses
StringEnumConverter(a built-in Newtonsoft converter). Either add an actual customJsonConverterexample or reword the prompt to avoid misleading setup.
prompt: |
I'm migrating our ASP.NET Core 8 project from Newtonsoft.Json to System.Text.Json. Here's a model class that uses Newtonsoft attributes and a custom converter. Convert this to System.Text.Json:
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
| | Using `JsonElement` after `JsonDocument` is disposed | JsonElement is invalid after dispose; clone with `element.Clone()` if needed | | ||
| | `[JsonIgnore]` from wrong namespace | Both Newtonsoft and System.Text.Json have `[JsonIgnore]` — wrong `using` = attribute ignored | | ||
| | Custom converter reading past the current token | System.Text.Json reader is strict — must read exactly the right tokens | | ||
| | `JsonExtensionData` with `Dictionary<string, object>` | Must be `Dictionary<string, JsonElement>` — not `object` or `JToken` | |
There was a problem hiding this comment.
This repeats the earlier claim that JsonExtensionData "must" use Dictionary<string, JsonElement> and that Dictionary<string, object> is invalid. System.Text.Json supports extension data on IDictionary<string, object> as well; if the goal is to prevent accidentally keeping JToken, rephrase this pitfall to focus on avoiding Newtonsoft types and explain the tradeoffs between object and JsonElement/JsonNode.
| | `JsonExtensionData` with `Dictionary<string, object>` | Must be `Dictionary<string, JsonElement>` — not `object` or `JToken` | | |
| | `JsonExtensionData` containing `JToken`/Newtonsoft types | Avoid `JToken`/`JObject` in extension data; use System.Text.Json types (`Dictionary<string, JsonElement>`/`JsonNode`) or `Dictionary<string, object>` with only CLR values | |
| scenarios: | ||
| - name: "Migrate model with Newtonsoft.Json attributes to System.Text.Json" | ||
| prompt: | | ||
| I'm migrating our ASP.NET Core 8 project from Newtonsoft.Json to System.Text.Json. Here's a model class that uses Newtonsoft attributes and a custom converter. Convert this to System.Text.Json: | ||
|
|
There was a problem hiding this comment.
This eval is being added under src/dotnet/tests/..., but the repo layout and evaluation workflow expect scenarios under tests/<plugin>/<skill>/eval.yaml and only trigger on tests/** (and plugins/**). Placing this under src/ means CI evaluation and CODEOWNERS folder validation won't run for the new scenario. Consider moving to tests/dotnet/migrating-newtonsoft-to-system-text-json/eval.yaml.
| options.DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull; // Common Newtonsoft setting | ||
|
|
There was a problem hiding this comment.
DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull is not Newtonsoft.Json's default (Json.NET includes nulls unless configured). Since this method is labeled "Match Newtonsoft.Json default behavior", consider separating true defaults from optional/common settings (or adjust the wording) so the guidance isn't misleading.
| options.DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull; // Common Newtonsoft setting | |
| // Additional commonly used Newtonsoft.Json-like settings (not defaults): | |
| options.DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull; // Common Newtonsoft setting (NullValueHandling.Ignore) |
| - "Changed [JsonExtensionData] Dictionary value type from JToken to JsonElement (critical difference!)" | ||
| - "Configured PropertyNameCaseInsensitive = true to match Newtonsoft default case-insensitive behavior" | ||
| - "Configured AllowTrailingCommas = true and NumberHandling = AllowReadingFromString for Newtonsoft compatibility" | ||
| - "Warned about behavioral differences (default PascalCase casing in STJ vs camelCase in Newtonsoft, strict parsing)" |
There was a problem hiding this comment.
The rubric's casing note is inconsistent with the skill doc: it expects a warning about "PascalCase in STJ vs camelCase in Newtonsoft". If you want to test casing differences, consider aligning this with the skill's own statements and/or explicitly scoping it to ASP.NET Core defaults vs library defaults to avoid grading correct answers as wrong.
| - "Warned about behavioral differences (default PascalCase casing in STJ vs camelCase in Newtonsoft, strict parsing)" | |
| - "Warned about behavioral differences (for example, stricter parsing behavior in System.Text.Json compared to Newtonsoft.Json)" |
| - "Configured PropertyNameCaseInsensitive = true to match Newtonsoft default case-insensitive behavior" | ||
| - "Configured AllowTrailingCommas = true and NumberHandling = AllowReadingFromString for Newtonsoft compatibility" | ||
| - "Warned about behavioral differences (default PascalCase casing in STJ vs camelCase in Newtonsoft, strict parsing)" | ||
| expect_tools: ["bash"] |
There was a problem hiding this comment.
expect_tools: ["bash"] forces the model to make at least one bash tool call or the scenario fails. This prompt is primarily a code-migration explanation and doesn't require shell usage, so this constraint may introduce unnecessary flakiness. Consider removing expect_tools (or only using it when the scenario includes a setup/fixture that actually needs bash).
| expect_tools: ["bash"] |
| **System.Text.Json (.NET 7+ — type discriminators):** | ||
| ```csharp | ||
| [JsonDerivedType(typeof(CreditCardPayment), typeDiscriminator: "credit")] | ||
| [JsonDerivedType(typeof(BankTransferPayment), typeDiscriminator: "bank")] | ||
| public abstract class Payment |
There was a problem hiding this comment.
The polymorphism section implies System.Text.Json will automatically emit a discriminator for [JsonDerivedType] types. STJ polymorphism requires explicit opt-in/configuration (e.g., JsonPolymorphic / JsonPolymorphismOptions / type-info resolver setup). Consider updating this snippet to show the required opt-in so readers don't assume the attributes alone are sufficient.
| ```skill | ||
| --- | ||
| name: migrating-newtonsoft-to-system-text-json | ||
| description: Migrate from Newtonsoft.Json to System.Text.Json, handling behavioral differences, custom converters, and common breaking changes. Use when converting a project from Newtonsoft.Json (Json.NET) to the built-in System.Text.Json serializer. | ||
| --- |
There was a problem hiding this comment.
These files are being added under src/dotnet/..., but the repo's documented layout and CI evaluation workflows expect skills under plugins/<plugin>/skills/<skill>/SKILL.md and scenarios under tests/<plugin>/<skill>/eval.yaml. As-is, the evaluation workflow (and CODEOWNERS folder validation) won't run because it only triggers on plugins/** and tests/**. Consider moving this skill to plugins/dotnet/skills/migrating-newtonsoft-to-system-text-json/ and the eval to tests/dotnet/migrating-newtonsoft-to-system-text-json/.
| static void ConfigureJsonOptions(JsonSerializerOptions options) | ||
| { | ||
| // Match Newtonsoft.Json default behavior: | ||
| options.PropertyNamingPolicy = JsonNamingPolicy.CamelCase; // Newtonsoft default | ||
| options.PropertyNameCaseInsensitive = true; // Newtonsoft default |
There was a problem hiding this comment.
The doc contradicts itself on default property naming: Step 1 says both Newtonsoft.Json and System.Text.Json default to PascalCase/as-declared, but Step 2 config sets PropertyNamingPolicy = CamelCase and labels it "Newtonsoft default". Please clarify the distinction (Json.NET library defaults vs ASP.NET Core defaults vs any custom ContractResolver/NamingPolicy) and make the guidance consistent.
| | `[JsonProperty(DefaultValueHandling = DefaultValueHandling.Ignore)]` | `[JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingDefault)]` | | ||
| | `[JsonConverter(typeof(MyConverter))]` | `[JsonConverter(typeof(MyConverter))]` (different base class!) | | ||
| | `[JsonConstructor]` | `[JsonConstructor]` (same name, different namespace) | | ||
| | `[JsonExtensionData]` | `[JsonExtensionData]` + must be `Dictionary<string, JsonElement>` (NOT `JToken`) | |
There was a problem hiding this comment.
[JsonExtensionData] doesn't have to be Dictionary<string, JsonElement> only—System.Text.Json also supports extension data on other shapes (e.g., IDictionary<string, object> and, in newer TFMs, JsonNode/JsonObject variants). If the intent is to recommend JsonElement as the closest replacement for JToken, consider rephrasing this as guidance rather than a hard requirement so the doc stays technically correct.
| | `[JsonExtensionData]` | `[JsonExtensionData]` + must be `Dictionary<string, JsonElement>` (NOT `JToken`) | | |
| | `[JsonExtensionData]` | `[JsonExtensionData]` + typically use `Dictionary<string, JsonElement>` as closest to `JToken` (other supported shapes are also valid) | |
| var settings = new JsonSerializerSettings | ||
| { | ||
| TypeNameHandling = TypeNameHandling.Auto // SECURITY RISK! | ||
| }; |
There was a problem hiding this comment.
This example configures JsonSerializerSettings with TypeNameHandling = TypeNameHandling.Auto, which is a known unsafe pattern when used with untrusted JSON because it lets an attacker control the concrete .NET type to instantiate, enabling gadget-based code execution or data tampering. Even though it is labeled as a security risk, including this as compilable example code may lead readers to copy it into production; consider replacing it with a non-usable anti-pattern illustration or a safer configuration that avoids TypeNameHandling for untrusted data.
| var settings = new JsonSerializerSettings | |
| { | |
| TypeNameHandling = TypeNameHandling.Auto // SECURITY RISK! | |
| }; | |
| // ❌ Insecure pattern shown for illustration only — DO NOT COPY THIS INTO PRODUCTION CODE. | |
| // The following configuration enables $type-based polymorphic deserialization, which is | |
| // vulnerable when used with untrusted JSON input: | |
| // | |
| // var settings = new JsonSerializerSettings | |
| // { | |
| // TypeNameHandling = TypeNameHandling.Auto | |
| // }; |
| options.Converters.Add(new JsonStringEnumConverter(JsonNamingPolicy.CamelCase)); | ||
|
|
||
| // Handle circular references (replaces PreserveReferencesHandling) | ||
| options.ReferenceHandler = ReferenceHandler.IgnoreCycles; // or Preserve for $ref/$id |
There was a problem hiding this comment.
Setting Preserve significantly increases the attack surface of JSON deserialization. Most call sites do not know how to properly reason about the increased attack surface and could find themselves in a vulnerable state. This is why S.T.J does not support $ref / $id in its default configuration.
It's fine if people want to set this, but they should do it only if they truly need it and have thought through the consequences of setting it. It shouldn't be promoted as a "try it and see if it solves your problem" mechanism. (What happens in practice is that somebody will set it, find that it doesn't solve their problem, then move on to some other diagnostic step without reverting this change.)
There was a problem hiding this comment.
To defend against this attack, the application should consider what happens if the adversary controls the edges in the object graph, not just the values of the nodes in the graph. For example, if you're deserializing a graph that represents a family tree, what happens if a node sets itself as its own parent? What happens if a node has two children, but both point to the same object in memory? (This last example isn't recursive; it's an example of a graph that has a many:1 relationship between edges and nodes.)
Both examples could violate business logic that the application otherwise tries to uphold. Allowing an untrusted client to control the edges in the graph represents a broader attack surface and involves more subtle reasoning compared to an untrusted client that can only control values within the graph.
| options.PropertyNamingPolicy = JsonNamingPolicy.CamelCase; // Newtonsoft default | ||
| options.PropertyNameCaseInsensitive = true; // Newtonsoft default | ||
| options.NumberHandling = JsonNumberHandling.AllowReadingFromString; // Newtonsoft coerces | ||
| options.ReadCommentHandling = JsonCommentHandling.Skip; // Newtonsoft allows |
There was a problem hiding this comment.
Similarly, this is a potentially dangerous setting and should not be encouraged unless the application has thought through the consequences. We chose not to allow this by default in S.T.J because it could lead to desynced deserialization attacks.
In these attacks, the frontend and the backend disagree on the contents of the payload. This could be because they disagree on where comments start or end, potentially allowing sensitive values to be smuggled through what appears to be an ignorable comment. (For example, Newtonsoft.Json, JSON5, and S.T.J all have different concepts on what "end of line" means for a single-line comment.)
There was a problem hiding this comment.
To defend against this attack, the application should:
- ensure that all deserializers in the system are operating in strict RFC compliance mode with the most restrictive settings (e.g., disallowing duplicate property entries in the payload); or
- ensure that all components involved in request handling use the same deserializer library, with the same major version and patch level, to ensure consistent payload processing; or
- ensure the frontend deserializes the payload to an object graph, then reserializes that object graph to a new payload before sending it on to the backend, so the backend sees only what the frontend believed the request to contain.
There was a problem hiding this comment.
I would add that case insensitive property handling (recommended a few lines above this) similarly makes systems susceptible to interoperability attacks.
While I get that such settings might be necessary to preserve a modicum of compatibility with Json.NET, at a bare minimum we should try to document security concerns in the skill. It should start off assuming the highest bar possible (using JsonSerializerDefaults.Strict) and then instruct the agent to interview the user about particular requirements that force the loosening of individual settings, e.g. "Should the serializer support comments? Should the serializer allow case insensitive property binding? Should it escape non-ASCII characters? Should it allow trailing commas? Should it allow duplicate JSON properties? etc.)
ViktorHofer
left a comment
There was a problem hiding this comment.
@mrsharm please fix the path. These files should be under plugins/... not src
| - No `serializer` parameter — use `options` and call `JsonSerializer.Serialize/Deserialize` for nested objects | ||
| - For polymorphic deserialization: use `JsonTypeInfo` and `[JsonDerivedType]` (.NET 7+) instead of custom type handling | ||
|
|
||
| ### Step 5: Replace JToken/JObject/JArray with JsonDocument/JsonElement |
There was a problem hiding this comment.
I think just always recommend JsonNode and friends. It's closer than JsonDocument/JsonElement.
|
|
||
| **System.Text.Json is NOT a drop-in replacement.** These behaviors differ by default: | ||
|
|
||
| | Behavior | Newtonsoft.Json | System.Text.Json | Impact | |
There was a problem hiding this comment.
STJ also escapes non-ASCII characters by default. Should recommend using relaxed encoder
There was a problem hiding this comment.
I think we should caveat any such recommendation with the trade-offs: escaped JSON strings are semantically equivalent to unescaped ones, so this should depend on how the output is being used. Will it be embedded in HTML? Is it fed to LLMs that might get confused by the escaping? etc.
eiriktsarpalis
left a comment
There was a problem hiding this comment.
It is an understatement to say that migrating serializers is a nontrivial task -- it is almost guaranteed that the new serializer is going to behave differently to an extent, so the goal of such a skill is to make sure all steps are taken so that risk is minimized.
An important aspect of minimizing risk in migrations of such scope is having a testing strategy. At a bare minimum, the agent should develop a test suite as it performs the migration comparing STJ serialization/deserialization behavior against the NJ baseline. This would help provide an empirically grounded feedback loop for the agent to apply necessary tweaks to DTOs, their attribute annotations, and their custom convert.
| ```bash | ||
| # Find all files using Newtonsoft attributes | ||
| grep -rn "using Newtonsoft.Json" --include="*.cs" | ||
| grep -rn "\[JsonProperty\|JsonConverter\|JsonIgnore\|JsonConstructor" --include="*.cs" |
There was a problem hiding this comment.
I'm pretty sure newer models can figure out regexes like that on their, so I would just skip that part for fear of it going out of sync.
…vement) Teaches migration from Newtonsoft.Json to System.Text.Json: attribute mapping differences, behavioral changes (casing, strictness, null handling), custom converter conversion, JToken->JsonElement/JsonNode migration, and polymorphic serialization with JsonDerivedType. Eval results: +13.8% improvement over baseline (threshold: 10%) Includes eval.yaml with migration scenario + negative test.
…LL.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…LL.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…LL.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…y structure - Move skill from src/dotnet/skills/ to plugins/dotnet/skills/ - Move eval from src/dotnet/tests/ to tests/dotnet/ - Add CODEOWNERS entries
2790f37 to
700e751
Compare
PR-FEEDBACK.md
Outdated
|
/evaluate |
Skill Validation Results
Model: claude-opus-4.6 | Judge: claude-opus-4.6 |
| /plugins/dotnet/skills/dotnet-aot-compat/ @agocke @dotnet/appmodel | ||
| /tests/dotnet/dotnet-aot-compat/ @agocke @dotnet/appmodel | ||
|
|
||
| /plugins/dotnet/skills/migrating-newtonsoft-to-system-text-json/ @mrsharm |
There was a problem hiding this comment.
Update these to the appropriate team.
|
now we have the new plugins factoring in main, this should not be in the general dotnet plugin. please move it to the dotnet-upgrade plugin which seems a better fit. |
Summary
Adds the migrating-newtonsoft-to-system-text-json skill for migrating from Newtonsoft.Json (Json.NET) to System.Text.Json.
Skill Validation Results — migrating-newtonsoft-to-system-text-json
Overall improvement: +10.8% (3 runs, not statistically significant)
Model: claude-opus-4.6 | Judge: claude-opus-4.6
3 Iterations.
What the Skill Teaches
Files